The Use of Residuals in Image Denoising

State-of-the-art image denoising algorithms attempt to recover natural image signals from their noisy observations, such that the statistics of the denoised image follow the statistical regularities of natural images. One aspect generally missing in these approaches is that the properties of the residual image (defined as the difference between the noisy observation and the denoised image) have not been well exploited. Here we demonstrate the usefulness of residual images in image denoising. In particular, we show that well-known full-reference image quality measures such as the mean-squared-error and the structural similarity index can be estimated from the residual image without the reference image. We also propose a procedure that has the potential to enhance the image quality of given image denoising algorithms.